We study transformer language models, analyzing attention heads whose attention patterns are spread out, and whose attention scores depend weakly on content. We argue that the softmax denominators of these heads are stable when the underlying token distribution is fixed. By sampling softmax denominators from a "calibration text", we can combine together the outputs of multiple such stable heads in the first layer of GPT2-Small, approximating their combined output by a linear summary of the surrounding text. This approximation enables a procedure where from the weights alone - and a single calibration text - we can uncover hundreds of first layer neurons that respond to high-level contextual properties of the surrounding text, including neurons that didn't activate on the calibration text.

Radio remains a pervasive medium for mass information dissemination, with AM/FM stations reaching more Americans than either smartphone-based social networking or live television. Increasingly, radio broadcasts are also streamed online and accessed over the Internet. We present WavePulse, a framework that records, documents, and analyzes radio content in real-time. While our framework is generally applicable, we showcase the efficacy of WavePulse in a collaborative project with a team of political scientists focusing on the 2024 Presidential Elections. We use WavePulse to monitor livestreams of 396 news radio stations over a period of three months, processing close to 500,000 hours of audio streams. These streams were converted into time-stamped, diarized transcripts and analyzed to track answer key political science questions at both the national and state levels. Our analysis revealed how local issues interacted with national trends, providing insights into information flow. Our results demonstrate WavePulse's efficacy in capturing and analyzing content from radio livestreams sourced from the Web. Code and dataset can be accessed at \url{https://wave-pulse.io}.

Harvey Castro talks about how AI could be used in cold cases and the symbiotic relationship between AI and a detective. Riverside, California, investigators linked a man's DNA to a 1979 cold case murder of a teenage girl, years after the same man passed a lie detector test about the crime, according to authorities. The body of 17-year-old Esther Gonzalez was found dumped in packed snow off Highway 243 in Banning, California, in 1979, and after an investigation, detectives determined the teen had been raped and bludgeoned to death. Last week, the Riverside County District Attorney's Office said in a press release that the case had been solved using forensic genealogy, over 45 years later. On Nov. 20, the Riverside County Regional Cold Case Homicide Team identified Lewis Randolph "Randy" Williamson, who died in 2014, as the killer. NEWS ANCHOR'S MYSTERIOUS DISAPPEARANCE WAS CRIME OF'JEALOUSY': PRIVATE INVESTIGATOR Gonzalez was attacked and murdered on Feb. 9, 1979, as she was walking to her sister's house in Banning from her parent's house in Beaumont.